feat: automatic Splunk version discovery by mkolasinski-splunk · Pull Request #214 · splunk/addonfactory-test-matrix-action

mkolasinski-splunk · 2026-06-25T15:18:42Z

Summary

Auto-discovers new Splunk major.minor versions (e.g. 10.4, 10.5) from Docker Hub and adds them to config/splunk_matrix.conf
Fetches end-of-support dates from Splunk's lifecycle page; falls back to UNKNOWN if unavailable
Prunes stanzas whose SUPPORTED date has passed (UNKNOWN stanzas are never auto-removed)
Keeps GENERAL.LATEST and GENERAL.OLDEST in sync with the actual version range
Regex boundary fix ensures 10.4 never accidentally matches 10.40
20 tests, 0 failures

What changes

splunk_matrix_update.py — 6 new functions + updated orchestrator:

Function	Purpose
`get_all_major_minor_versions`	Extract unique `X.Y` prefixes from Docker Hub tags
`get_new_versions`	Find prefixes not yet in config
`get_supported_date`	Scrape Splunk lifecycle page for EOL date
`add_new_version_stanza`	Create a complete new `[X.Y]` stanza
`remove_expired_versions`	Prune past-EOL stanzas
`update_general_section`	Sync `LATEST`/`OLDEST`

requirements-test.txt — new file; pytest moved here from requirements.txt

Test plan

Review config/splunk_matrix.conf diff in the weekly auto-PR to confirm new versions are picked up correctly
Verify SUPPORTED = UNKNOWN stanzas are not auto-pruned
Confirm GENERAL.LATEST and GENERAL.OLDEST reflect actual version range after run

🤖 Generated with Claude Code

Note on auto-discovery behaviour: versions that are currently in the Docker Hub tag list but have SUPPORTED = Not Released on Splunk's lifecycle page (e.g. 10.1) are skipped today because their EOL date is unknown. Once such a version receives a GA/EOL date it will be automatically added on the next weekly run. This is intentional — reviewers of future auto-PRs should expect to see previously-absent stanzas appear when Splunk publishes a new GA date.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add get_all_major_minor_versions() and get_new_versions() to discover new Splunk major.minor versions from Docker Hub. Include test scaffold with 4 tests covering discovery logic. Add pytest to requirements. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Adds add_new_version_stanza() function to create config stanzas for newly discovered Splunk versions. Returns True if stanza was added, False if version is expired or has no Docker image. Always sets PYTHON39=true and PYTHON37=false on new stanzas. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ests

Implements remove_expired_versions() to prune stanzas with expired support dates (skipping UNKNOWN entries) and update_general_section() to sync LATEST/OLDEST values based on semver ordering. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Replaces the body of update_splunk_version() to call get_new_versions, add_new_version_stanza, remove_expired_versions, and update_general_section in addition to the existing patch-version update loop. Adds integration test covering new-minor discovery, patch bump, and GENERAL sync end-to-end. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 15dcaefc3a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-06-25T15:23:06Z

+    config.set(major_minor, "VERSION", latest_version)
+    if build:
+        config.set(major_minor, "BUILD", build)
+    config.set(major_minor, "SUPPORTED", supported)


Avoid writing UNKNOWN support dates

When get_supported_date() returns UNKNOWN for a scrape/network failure or an unlisted version, this still commits a new stanza with SUPPORTED = UNKNOWN. The Docker action consumer does not accept that value: _generate_supported_splunk() in addonfactory_test_matrix_action/main.py unconditionally runs datetime.strptime(supported_splunk_string, "%Y-%m-%d"), so any weekly auto-discovery PR that adds such a stanza will make the action crash for all workflows reading the matrix. This is especially likely because the current Splunk policy page uses dates like May 18 2028 rather than the comma format the scraper expects.

Useful? React with 👍 / 👎.

Fixed in commit a1a8a6e.

add_new_version_stanza now returns False immediately when get_supported_date returns UNKNOWN — no stanza is written, so the consumer's unconditional strptime can never receive an invalid value.

The skip condition was also tightened to eol_date <= today (i.e. reject on the EOL day itself) to align with main.py's today >= eol check. The test that previously asserted the stanza was added with SUPPORTED = UNKNOWN has been renamed and flipped to assert it is skipped; two new boundary-day tests were added.

…ated month format The page uses <td>X.Y</td><td>RELEASE</td><td>EOL</td> rows with dates formatted as "May 18 2028" (%b %d %Y), not full month names with a comma. The previous regex also captured the release-date column instead of the EOL-date column. Updated tests to use the real HTML table structure. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- add_new_version_stanza now returns False for UNKNOWN supported dates (main.py does an unconditional strptime, so UNKNOWN would crash the action) - Both add_new_version_stanza and remove_expired_versions now use <= today to match main.py's `today >= eol` semantics; on the exact EOL day the consumer already skips the stanza, so the pruner should remove it too - Renamed test to reflect the new behaviour (skips, not adds, on UNKNOWN) - Added boundary-day tests for both add_new_version_stanza and remove_expired_versions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mkolasinski-splunk · 2026-06-25T15:50:33Z

P2 — Prune versions on their EOL date: addressed in commit a1a8a6e

The boundary-date alignment has been fixed across the board:

remove_expired_versions: comparison changed from < today to <= today — a stanza is now removed on its exact EOL day, matching main.py's today >= eol skip condition.
add_new_version_stanza: skip condition similarly updated to eol_date <= today so a newly-discovered version whose EOL is today is never written to the file.

This closes the one-day window where LATEST/OLDEST could point to a stanza that the consumer was already skipping.

New tests added:

test_remove_expired_versions_removes_on_exact_eol_day — verifies the pruner removes on today's date
test_add_new_version_stanza_skips_on_exact_eol_day — verifies the adder rejects on today's date

mkolasinski-splunk · 2026-06-25T15:54:53Z

@codex please review

These are internal planning artefacts and should not be part of the PR. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a1a8a6ec88

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-06-25T16:01:07Z

+    if build:
+        config.set(major_minor, "BUILD", build)


Require BUILD before adding a new stanza

When Docker Hub exposes a version tag but the matching 12-character build tag is absent from the fetched tag list, this branch still writes a new [X.Y] section without BUILD. The runtime matrix generator later does props["build"] in _generate_supported_splunk, so the next released action using that generated config fails with a KeyError instead of just skipping or deferring that version. Please only add the stanza when a build was found, or provide a safe fallback.

Useful? React with 👍 / 👎.

Fixed in commit 30516e3.

add_new_version_stanza now returns False when get_build_number returns None — the stanza is only written when a build hash is found. BUILD is always set (the conditional if build branch was removed). A new test test_add_new_version_stanza_skips_when_no_build_hash covers this path.

chatgpt-codex-connector · 2026-06-25T16:01:07Z

+    conf_path = tmp_path / "splunk_matrix.conf"
+    conf_path.write_text(
+        "[GENERAL]\nLATEST = 9.3\nOLDEST = 9.3\n"
+        "[9.3]\nVERSION = 9.3.10\nBUILD = aabbccddee00\nSUPPORTED = 2026-07-24\n"


Use a non-expiring fixture date

This fixture expires on 2026-07-24; beginning that day, update_splunk_version() will remove the [9.3] stanza via the new pruning step before the assertions that 9.3 remains and is OLDEST. That makes the test suite start failing solely because the calendar advanced, so this should use a dynamically future date.

Useful? React with 👍 / 👎.

Fixed in commit 30516e3.

The hardcoded SUPPORTED = 2026-07-24 date has been replaced with today + 10 years, computed at test runtime, so the pruner will never remove the 9.3 stanza as time advances. The integration test also gained a BUILD assertion for the new 10.4 stanza.

Runs tests/test_splunk_matrix_update.py on every PR and push. build_release now depends on both pre-commit and test passing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…date - add_new_version_stanza now returns False when no build hash is found — main.py accesses props["build"] unconditionally, so a missing BUILD key would cause a KeyError at action runtime - Integration test SUPPORTED date changed to today+10y so the pruner never removes the 9.3 stanza as the calendar advances - Added test_add_new_version_stanza_skips_when_no_build_hash to cover the new guard; added BUILD assertion to integration test for new 10.4 stanza Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mbruzda-splunk · 2026-06-26T11:41:03Z

A couple of review notes (non-blocking, but worth addressing):

1. Silent failure if the scrape breaks. The whole discovery path rests on get_supported_date's regex matching Splunk's lifecycle-page HTML exactly (<td>X.Y</td>\s*<td>…</td>\s*<td>EOL</td> — no attributes/classes, exact column order). I verified it matches the live page today and the dates line up with config/splunk_matrix.conf exactly. But if Splunk restyles that page, get_supported_date → "UNKNOWN" → add_new_version_stanza returns False, so new versions silently stop being discovered with zero signal. There's no logging in the module. Suggest a print(..., file=sys.stderr) when a version that does have a Docker image fails its date lookup, so the degraded state shows up in the weekly job logs instead of just producing a quietly-incomplete PR.

2. Auto-discovery will re-add deliberately-omitted versions. The config intentionally has 10.0 and 10.2 but not 10.1. That's safe today only because 10.1's EOL is currently "Not Released" → UNKNOWN → skipped. Once such a version gets a GA/EOL date it'll be auto-added on the next run. That may well be the intended behavior for auto-discovery — just flagging it as a behavior change worth a sentence in the PR description so reviewers of the weekly auto-PR aren't surprised by a reappearing stanza.

If get_supported_date returns UNKNOWN for a newly discovered version the weekly workflow now fails visibly instead of silently skipping it. This ensures a broken lifecycle page scraper doesn't produce a quietly incomplete auto-discovery PR. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mkolasinski-splunk · 2026-06-26T11:52:55Z

Thanks for the review.

1. Silent scrape failure — fixed in commit 1ae5672. update_splunk_version now calls get_supported_date before attempting to add each new version. If it returns UNKNOWN, it prints an error to stderr and calls sys.exit(1), which makes the weekly workflow step fail red. That way a broken scraper (e.g. Splunk restyling the lifecycle page) is immediately visible rather than producing a quietly incomplete PR.

2. Re-adding deliberately-omitted versions — agreed this is intended behaviour: if a version gets a GA date it should be auto-added. I've added a note to the PR description so future reviewers of the weekly auto-PR aren't surprised by a reappearing stanza.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

srv-rr-github-token · 2026-06-26T13:23:07Z

🎉 This PR is included in version 3.2.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

mkolasinski-splunk and others added 10 commits June 25, 2026 15:29

docs: add design spec for automatic Splunk version discovery

301cb5e

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs: add implementation plan for Splunk auto-discovery

22a852e

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add lifecycle page scraper for SUPPORTED dates

c5df44c

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

test: assert BUILD field and return value in add_new_version_stanza t…

9a90323

…ests

test: assert BUILD field updated in orchestration integration test

170684b

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: add version boundary to lifecycle regex and improve test coverage

15dcaef

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mkolasinski-splunk requested a review from a team as a code owner June 25, 2026 15:18

chatgpt-codex-connector Bot reviewed Jun 25, 2026

View reviewed changes

mkolasinski-splunk and others added 3 commits June 25, 2026 17:25

style: apply black formatting

81fa7bc

chore: remove plan and spec docs from branch

d5d68fd

These are internal planning artefacts and should not be part of the PR. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chatgpt-codex-connector Bot reviewed Jun 25, 2026

View reviewed changes

mkolasinski-splunk and others added 2 commits June 25, 2026 18:06

ci: add pytest job to main workflow

0832c80

Runs tests/test_splunk_matrix_update.py on every PR and push. build_release now depends on both pre-commit and test passing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mbruzda-splunk previously approved these changes Jun 26, 2026

View reviewed changes

mkolasinski-splunk dismissed mbruzda-splunk’s stale review via 1ae5672 June 26, 2026 11:52

style: apply black formatting

ed1f57c

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mbruzda-splunk approved these changes Jun 26, 2026

View reviewed changes

mkolasinski-splunk merged commit b211499 into main Jun 26, 2026
4 checks passed

mkolasinski-splunk deleted the feat/splunk-auto-discovery branch June 26, 2026 13:21

srv-rr-github-token added the released label Jun 26, 2026

Uh oh!

Conversation

mkolasinski-splunk commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changes

Test plan

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

mkolasinski-splunk Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

mkolasinski-splunk commented Jun 25, 2026

Uh oh!

mkolasinski-splunk commented Jun 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

mkolasinski-splunk Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

mkolasinski-splunk Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

mbruzda-splunk commented Jun 26, 2026

Uh oh!

mkolasinski-splunk commented Jun 26, 2026

Uh oh!

Uh oh!

srv-rr-github-token commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mkolasinski-splunk commented Jun 25, 2026 •

edited

Loading